The  AAFP  Patient  Safety  Reporting  System: 
Deveiopment  and  Legai  issues  Pertinent  to 
Medicai  Error  Tracking  and  Anaiysis 

Robert  L,  Phillips,  Susan  M,  Dovey,  John  S,  Hickner, 

Deborah  Graham,  Michele  Johnson 


Abstract 

The  Institute  of  Medieine  makes  the  ease  that  patient  safety  data  are  a  critieal 
input  for  redesigning  eare  proeesses  in  ways  that  will  make  health  care  safer. 
Mandatory  and  voluntary  error-reporting  systems  are  sources  of  such  data. 
However,  a  chasm  of  legal  and  practical  problems  exists  between  the  collection  of 
error  reports  and  responding  to  reporting  providers  to  improve  the  quality  and 
safety  of  the  systems  in  which  they  work.  Between  2000  and  2004,  the  American 
Academy  of  Family  Physicians  (AAFP)  developed  and  tested  a  voluntary  error¬ 
reporting  system.  In  this  paper  we  discuss  the  current  design  of  the  AAFP’s 
system  and  the  legal  and  practical  constraints  that  stand  in  the  way  of  its 
becoming  a  more  robust  quality-improvement  tool.  We  explain  decisions  to  make 
the  reporting  system  Web-based  (rather  than  paper-based),  to  be  anonymous 
(rather  than  confidential),  to  not  provide  direct  or  specific  feedback  to  reporters, 
and  to  make  it  capable  of  receiving  reports  of  both  “sentinel”  events  and 
“intensive”  reporting.  This  paper  will  clarify  what  is  currently  done  with  error 
reports  and  how,  despite  current  limitations,  the  reporting  system  informs  and 
promotes  a  variety  of  other  quality  initiatives  of  the  AAFP.  We  also  highlight  how 
this  reporting  system  could  more  robustly  improve  patient  safety  and  quality  in 
health  care  if  legislative  and  other  remedies  are  implemented  to  bridge  the 
existing  chasm. 


Introduction 

In  To  Err  Is  Human,  the  Institute  of  Medicine  (lOM)  recognized  that  error¬ 
reporting  systems  had  dual,  but  simultaneously  difficult  functions  of  increasing 
accountability  and  providing  information  that  may  lead  to  safer  health  care.^  In 
lOM’s  estimation,  mandatory  reporting  systems  lend  themselves  to  the  former, 
and  voluntary  reporting  systems  to  the  latter.  The  lOM  recommended  funding  and 
support  of  voluntary  reporting  efforts,  specifically  recognizing  the  need  for 
confidentiality  provisions,  reporter  feedback  mechanisms,  and  improved 
understanding  of  how  reported  information  is  used.  Leaders  of  the  patient  safety 
movement  agree  that  voluntary  reporting  systems  can  measurably  improve  safety 
if  reporting  is  protected  against  discovery  and  provides  reporters  with  useful 
information  from  expert  analysis.  They  also  suggest  that  the  complexity  and 
expense  of  running  a  national  voluntary  error-reporting  system  would  be 
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overwhelming,  and  that  specialty-based  or  system-specific  reporting  programs  are 
more  feasible. 

Researchers  at  the  Robert  Graham  Center,  a  research  center  and  division  of 
the  American  Academy  of  Family  Physicians  (AAFP),  seized  upon  To  Err  Is 
Human  and  its  recommendations  for  voluntary  reporting.  The  Graham  Center 
staff  felt  compelled  to  turn  its  research  focus  to  patient  safety,  due  to  the  scarcity 
of  information  about  threats  to  patient  safety  in  outpatient  settings,  where  most 
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Americans  receive  most  health  care.  It  was  decided  that  an  early  pilot  study  was 
needed  to  see  what  threats  to  patient  safety  family  and  general  physicians  would 
recognize  and  report,  and  whether  it  made  a  difference  if  they  reported  by  a  Web- 
or  paper-based  system.  This  initial  study,  launched  in  2000,  led  to  three 
subsequent  studies  within  the  AAFP  National  Network  for  Family  Practice  and 
Primary  Care  Research  (National  Network)  and  with  collaborators  in  six  other 
countries.  These  studies  have  helped  refine  a  specialty-based,  Web-enabled  error¬ 
reporting  tool  and  have  demonstrated  its  utility  for  identifying  threats  to  the  safety 
of  patients  who  enter  the  primary  care  system."^  This  paper  is  about  this  error¬ 
reporting  tool — its  evolution,  the  data  it  yields,  reporters’  experience  with  it,  ways 
that  it  could  contribute  to  other  reporting  programs,  the  legal  and  practical  barriers 
to  its  full  utilization,  and  the  potential  for  its  role  in  a  broader  patient  safety 
agenda  if  barriers  are  lifted. 


Background 

Family  practice  is  complex,  characterized  by  customized  care  that  responds  to 
individual  patients’  needs,  values,  and  preferences  across  a  broad  spectrum  of 
health  and  illness  concerns. Its  diversity,  scope,  and  variation  in  structure  and 
infrastructure  may  offer  more  opportunity  for  error  than  hospital-based  care, 
which  tends  to  be  more  highly  regulated  and  procedure  oriented.  Making  primary 
care  safer  for  patients  is  complicated  by  a  lack  of  understanding  of  the  nature  and 
distribution  of  errors  that  occur  in  this  setting  and  that  are  amenable  to  systematic 
change.  As  well,  many  primary  care  physicians  are  only  tenuously  connected  to 
other  providers  and  may  not  be  recognized  as  part  of  some  health  care  delivery 
systems.^  Malpractice  data  reveal  that  most  error-related  claims  for  primary  care 
physicians  come  from  care  delivered  outside  of  hospitals,  and  suggest  that 
seemingly  trivial  errors  can  severely  injure  and  kill  people.^ 

Prior  to  our  own  research,  only  two  studies  had  explored  the  epidemiology  of 
errors  in  primary  care.  One,  an  Australian  study,  used  incident  reports  to  describe 
events  that  resulted,  or  could  have  resulted,  in  harm  to  patients. These  events 
were  not  necessarily  errors.  The  other  study  reviewed  a  risk  management 
database,  applying  a  taxonomy  developed  from  review  of  hospital  records.  The 
growing  body  of  evidence,  and  the  lOM  patient  safety  report  series,  revealed  that 
the  health  system  could  cause  substantial  harm,  alerting  the  U.S.  public  and 
politicians  of  the  need  to  protect  people  from  safety  threats.  Primary  care 
providers,  including  family  physicians,  had  so  far  largely  been  excused  from 
engaging  in  discussions  about  medical  errors.  While  many  important  initiatives  in 
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hospitals  offered  hope  for  greater  patient  safety,  other  equally  important 
opportunities  were  being  overlooked  because  they  existed  outside  of  hospitals. 


The  AAFP  error-reporting  system: 

AAFP  Patient  Safety  Reports 

In  developing  the  AAFP  Patient  Safety  Reports,  we  attempted  to  design  in  the 
key  characteristics  of  voluntary  error-reporting  systems  proposed  by  safety 
experts  to  the  degree  possible.^’  These  key  characteristics  are  that  the  reporting 
systems  be  nonpunitive  and  undiscoverable,  confidential,  timely  and  responsive, 
and  easy  to  use.  As  recommended  by  the  lOM,  we  designed  a  reporting  system 
that  could  support  causal  analysis,  including  free-text  descriptions  of  error  events, 
guided  by  a  standard  set  of  questions.  Also  in  keeping  with  recommendations, 
we  developed  a  core  group  of  independent  peer  experts  to  analyze  the  error 
reports. 

AAFP  error-reporting  research 

The  AAFP  error-reporting  system  and  the  International  Taxonomy  of  Medical 
Errors  in  Primary  Care  have  been  developed  and  revised  based  on  four  research 
studies  carried  out  in  family  physicians’  offices  between  2000  and  2004. 

Our  first  study,  completed  in  January  2001,  taught  us  that  physicians 
recognized  many  errors  and  were  willing  to  report  them,  even  errors  with  serious 
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adverse  patient  outcomes.  Fifty  physicians  of  the  AAFP  National  Research 
Network  were  randomly  assigned  to  either  arm  of  a  crossover  trial  to  test  whether 
there  were  differences  in  reporting  by  paper  or  computer.  This  study  was  an  early 
foray  into  describing  the  spectrum  of  errors  recognized  by  physicians  in  primary 
care.  The  paper  system  included  a  pocket-sized  paper  card  with  prompts  to  report 
data  items,  and  the  computer  reporting  system  had  the  same  data  prompts.  Errors 
were  defined  as  events  in  physicians’  practices  that  should  not  have  happened  and 
were  not  anticipated.  Physicians  were  encouraged  to  identify  an  incident  “that 
should  not  happen  in  my  practice  and  I  don’t  want  it  to  happen  again.”  These 
could  be  administrative  or  clinical  errors.  They  could  be  adverse  events  (events 
resulting  in  unintended  harm)  or  near  misses  (events  that  had  the  potential  to 
cause  harm).  The  main  outcome  measures  were  error  category,  error  rate,  and 
error  consequence. 

Eorty-two  physicians  reported  343  incidents  over  10  weeks.  Thirteen  reports 
(4  percent)  were  of  adverse  events  not  arising  from  an  error,  283  (83  percent) 
were  of  mistakes  in  care  arising  from  system  and  process  failures,  and  47 
(14  percent)  were  of  errors  in  clinical  judgment.  Eurther  breakdown  of  these 
errors  is  given  in  Table  I.  Based  on  data  from  paper  reports  only,  the  error-report 
rate  in  this  study  was  45  per  doctor  per  year,  or  about  I  error  per  working  week. 
There  was  no  clinical  or  cost  consequence  in  54  percent  of  the  error  reports. 
Appropriate  care  was  delayed  or  extended  in  22  percent  of  reports,  and  additional 
financial  costs  were  reported  in  18  percent.  Nine  of  the  reported  errors 
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Table  1.  Distribution  of  error  types  from  three  AAFP  error-report  studies 


Error  types 

AAFP  1st  study 
(n  =  330) 

%  error  reports 

International 
(n  =  429) 

%  error  reports 

AAFP  2nd  study* 
(n  =  838) 

%  error  codes 

Process  errors 

86 

79 

96 

Office  administration 

31 

19 

51 

Filing  system 

12 

9 

Chart  complete/available 

8 

19 

Patient  fiow 

2 

6 

Message  handling 

4 

3 

Appointments 

3 

12 

Investigations 

25 

17 

13 

Treatments 

23 

26 

16 

Medications 

17 

14 

Communications 

6 

14 

13 

Payment 

1 

1 

3 

Workforce  management 

2 

1 

Knowiedge  and  skiiis 

14 

21 

4 

‘More  than  one  error  could  be  coded  for  each  report  in  this  study. 


precipitated  hospital  admission,  and  one  resulted  in  death.  No  judgment  about  the 
severity  of  these  consequences  was  sought  from  reporters  or  constructed  by  the 
research  team  in  this  initial  study.  It  seems  reasonable  to  suppose  that  a  death — 
the  result  of  a  mishandled  phone  message — was  the  most  severe  outcome. 

Legal  and  practical  issues  in  preliminary  studies 

There  were  few  legal  precedents  regarding  discoverability  of  physician  error 
reports  in  1999.  We  chose  to  use  ethical  review  and  approval  by  an  institutional 
review  committee  as  the  standard  for  protecting  participating  physicians.  In  the 
research  design,  we  used  many  safeguards  to  protect  the  identity  of  the  reporting 
physicians.  We  destroyed  any  documentation  that  could  link  error  reports  to 
individual  physician  reporters.  Documents  naming  physician  participants  were 
meticulously  shredded  and  computer  fdes  destroyed,  leaving  only  data  related  to 
the  reported  error.  These  protections  made  it  impossible  to  do  root-cause  analysis 
or  to  provide  individualized  feedback.  Individual  reports  were  never  made 
publicly  available,  and  publications  from  this  study  include  only  aggregate 
analyses  of  reports.  ’  On  occasion,  for  training  purposes,  example  cases 
combining  the  characteristics  of  two  or  more  reports  were  discussed  outside  of  the 
research  team.  Despite  efforts  to  guarantee  anonymity  of  the  reporting  physicians, 
however,  some  participants  in  this  study  are  identifiable.  Before  publishing  the 
paper  reporting  the  study  results,  we  asked  participants  if  they  wished  to  be 
named  in  the  “Acknowledgments”  section  of  the  paper,  and  many  elected  to  be 
named — choosing  to  make  their  participation  in  studying  patient  safety  a  matter 
of  public  record. 
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International  research 

The  first  AAFP  study  led  to  a  Commonwealth  Fund-supported  study  of  errors 
reported  by  physieians  in  seven  eountries:  the  United  States,  Canada,  New 
Zealand,  England,  the  Netherlands,  Australia,  and  Germany/"^  The  eomputerized 
error-reporting  system  was  tweaked  to  improve  its  funetion  and  no  paper  reports 
were  made,  beeause  the  first  study  provided  reassurance  that  the  types  of  errors 
reported  by  paper  and  computer  would  probably  be  similar.  Reports  remained 
anonymous  because  discoverability  was  not  protected  in  every  country. 

This  study  involved  more  than  100  family  and  general  physicians  in  the 
7  countries,  and  they  collectively  reported  more  than  600  errors  over  a  12-week 
period  in  2001.  The  distribution  of  error  types  was  similar  to  the  previous  U.S. 
study.  However,  there  were  some  differences  associated  with  countries  with 
greater  computerization  of  practice  (New  Zealand,  England).  Most  of  the  reports 
were  related  to  process  problems  (79  percent),  including  treatments  errors 
(26  percent),  mistakes  in  office  administration  (19  percent),  and  communication 
problems  (14  percent)  (Table  1). 

This  Primary  Care  International  Study  of  Medical  Errors  (PCISME)  spawned 
the  LINNAEUS  (Learning  in  an  International  Network  About  Errors  and 
Understanding  Safety)  Collaboration,  which  continues  to  work  together  and  its 
members  have  become  recognized  patient  safety  research  experts  in  their  own 
countries.  With  the  PCISME  study,  the  computerized  error-reporting  system 
was  tested  around  the  world  in  English  and  German  and  yielded  important 
information  about  medical  errors  in  primary  care.'"^ 

Legal  and  practical  issues  raised  by  international  research 

This  study  raised  new  legal  and  practical  issues.  Data  were  reported  in  an 
electronic  system  developed  in  conjunction  with  a  British  software  developer, 
who  leased  server  space  from  a  British  Government  agency  for  the  central  data 
repository.  During  report  processing,  however,  the  physical  server  holding  study 
data  at  any  particular  time  could  be  in  any  of  a  number  of  locations  throughout  the 
world.  It  was  not  the  data  themselves,  but  access  to  the  data  that  was  at  issue.  This 
is  an  important  issue  that  may  deserve  further  consideration  by  experts  in 
international  law:  What  access  might  third  parties  (such  as  malpractice  lawyers) 
have  to  error  report  data  that  are  not  held  in  their  own  country — including  data 
from  reporters  in  other  countries,  operating  under  different  legal  codes?  Some  of 
the  countries  involved  in  our  study  have  bilateral  national  agreements,  but  others 
do  not.  There  is  a  body  of  legal  precedent,  for  example,  of  American  legal  codes 
failing  to  have  effect  in  New  Zealand.  And  what  is  to  be  done  about  accessing 
data  that  have  no  known  country  of  “residence”  but  exist  only  “virtually,”  as 
electronic  data  might?  International  studies  and  even  international  error-reporting 
systems  are  likely  to  be  characterized  by  a  unique  set  of  legal  issues  that  have  so 
far  not  been  addressed.  Currently,  most  efforts  are  aimed  at  establishing  legal 
frameworks  for  reporting  systems  within  nations,  and  not  among  them. 
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Focused  domestic  research 

After  the  first  two  studies,  the  AAFP  reeeived  support  for  a  Developmental 
Center  of  Excellence  for  Research  in  Patient  Safety  (DCERPS)  from  the  Agency 
for  Healthcare  Research  and  Quality  (AHRQ)  (Grant  Number  5  P20  HSOl  1584) 
and  a  supplemental  patient  safety  research  grant  (AHRQ  Grant  Number  5  R21 
HSO  13554).  These  grants  were  an  important  next  step  for  AAEP’s  patient  safety 
agenda  and  competencies.  Among  other  things,  they  permitted  formalization  of 
the  error-reporting  system  as  an  in-house,  Web-based  process. 

In  2003,  the  DCERPS  launched  two  error-reporting  studies  in  the  National 
Network  to  address  questions  raised  in  the  first  two  studies.  We  hypothesized  that 
physicians  might  identify  different  errors  than  their  staff  and  patients.  Eamily 
physicians,  their  staffs,  and  their  patients  from  10  clinics  across  the  United  States 
were  asked  to  submit  error  reports  continuously  during  a  10-week  period, 
including  5  intensive  reporting  days  during  which  they  were  to  report  every  error 
they  witnessed  rather  than  just  those  they  thought  were  important  to  report,  using 
the  same  error  definition.  Physicians  and  staffs  were  given  the  option  of  reporting 
via  the  Internet  to  the  AAEP  Patient  Safety  Reports  secure  Web  site  or  via  written 
reports.  Patients  had  the  same  options  and  also  an  automated  telephone  reporting 
system.  The  participants  submitted  838  reports  (Table  1).  The  overwhelming 
majority  of  reports  were  made  by  the  Web-based  system  for  all  reporting  groups. 

Legal  and  practical  issues  raised  by  DCERPS  research 

Eederal  funding  carries  legal  protections  from  discoverability.  However, 
because  these  legal  protections  had  never  been  tested,  we  maintained  the 
anonymity  of  reporters.  We  did  so  over  protests  from  our  reporting  physicians, 
who  would  have  preferred  receiving  specific  feedback.  We  assured  anonymity  by 
using  reporter  codes  specific  only  to  reporter  types,  not  individuals,  and  by 
removing  “cookies”  and  other  report  evidence  from  reporters’  computers  with 
each  submission.  To  enhance  data  security  and  the  protection  of  reporters,  the 
AAEP  invested  in  more  robust  firewalls  within  the  error-report  server,  and 
separated  report  data  from  the  reporting  process  server  the  moment  reports  were 
submitted.  The  AAFP  funded  a  formal,  external  test  of  the  system’s  security 
before  implementing  the  studies. 

Summary  of  quantitative  findings 
from  error-reporting  studies 

Table  1  summarizes  the  distribution  of  the  types  of  errors  in  the  preliminary 
AAFP  study,  the  international  study,  and  the  DCERPS  study — the  three  primary 
care  patient  safety  studies  described  above.  Process  errors  (83  percent,  79  percent, 
and  96  percent,  respectively)  surpassed  knowledge  and  skill  errors  (14  percent, 

21  percent,  and  4  percent,  respectively)  in  all  three  studies.  The  most  commonly 
reported  health  process  errors  in  all  three  studies  were  office  administration  errors 
(31  percent,  19  percent,  and  51  percent,  respectively)  and  treatment  errors 
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(23  percent,  26  percent,  and  16  percent,  respectively).  Investigation  errors 
(25  percent,  17  percent,  and  13  percent,  respectively)  were  also  common. 

Several  of  the  most  severe  outcomes  resulted  from  seemingly  innocuous  and 
common  communication  errors.  Most  notable,  the  mishandling  of  messages  from 
patients  was  associated  with  two  deaths  and  a  fetal  demise. 

Differences  in  study  design  and  analysis  may  explain  some  of  the  differences 
between  studies.  Errors  in  the  first  two  studies  were  reported  only  by  physicians, 
but  errors  in  the  third  study  were  reported  by  physicians,  staffs,  and  patients. 
(Table  1  includes  physician  and  staff  data  only.)  In  addition,  the  coding 
procedures  used  in  the  second  U.S.  AAFP  study  (the  DCERPS  study)  differed 
slightly  from  those  used  in  the  first  two  studies.  In  the  first  AAFP  study  and  in  the 
international  study,  only  one  error  code  was  assigned  to  each  error  report.  If  more 
than  one  error  was  reported,  only  the  code  corresponding  to  the  first  error  was 
assigned  to  the  report.  The  first  two  studies  taught  us  that  understanding  how 
errors  cascade  was  important.  As  a  result,  in  the  second  U.S.  AAFP  study,  all 
errors  were  coded.  In  addition,  some  errors  could  be  assigned  more  than  one  code 
(e.g.,  an  error  could  be  assigned  both  an  investigation  error  code  and  a 
communication  error  code).  Consequently,  the  percentages  included  in  Table  1  for 
columns  two  and  three  reflect  the  percent  of  error  reports,  while  the  percentages 
for  column  four  reflect  the  percent  of  error  codes. 

Qualitative  findings  from  AAFP  error-report  studies 

The  supplemental  AHRQ  grant  to  the  DCERPS  supported  a  study  of  testing 
processes  (laboratory,  radiology)  that  included  incident  reporting,  as  well  as  focus 
groups  and  human  systems  engineering  clinic  evaluations.  The  study  focus 
included  all  steps  involved  in  ordering,  receiving,  and  responding  to  patient 
investigations.  The  aim  was  to  describe  testing  process  errors  recognized  by 
family  physicians  and  their  office  staffs,  to  investigate  causes  and  consequences 
of  these  errors,  and  to  generate  hypotheses  about  interventions  that  might  avert 
testing  process  errors.  Eight  practices  were  to  report  these  errors  over  a  48-week 
period  in  2003.  This  was  to  be  followed  by  qualitative  focus  groups  in  each 
practice,  and  human  systems  engineer  evaluations  in  two  of  the  practices. 

Participants  submitted  661  testing  process  errors,  but  the  study  ended  early,  in 
week  32,  because  reporting  significantly  tapered  off.  The  error  reports  have  not 
yet  been  coded  and  analyzed. 

Focus  groups  conducted  as  part  of  this  research  revealed  that  participants 
found  it  easy  to  make  reports,  usually  taking  only  1-3  minutes.  Major  barriers  to 


*  Two  patients  committed  suicide  after  calling  to  say  they  were  very  depressed  and  considering  suicide;  both 
times  their  messages  were  not  directly  delivered  to  the  physicians  but  simply  placed  on  the  doctors’  desks, 
where  they  were  found  too  late.  The  fetal  demise  occurred  when  a  term-pregnant  patient  who  called  about  not 
feeling  her  baby  move  was  not  appropriately  triaged  by  a  junior  nurse  covering  the  phones  over  lunch,  and 
the  message  of  the  patient’s  phone  call  was  placed  in  a  stack  of  charts.  The  patient  then  sought  help  at  an 
urgent  care  center,  where  she  was  forced  to  wait,  only  to  be  referred  back  to  her  own  physician  without 
evaluation.  By  the  time  the  patient  finally  reached  her  physician’s  office,  it  was  closed.  She  went  to  the 
emergency  room,  where  she  received  the  tragic  news. 
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reporting  were  remembering  to  report  and  other  time  eommitments,  sueh  as 
paperwork  and  elinieal  responsibilities.  Many  of  the  foeus  group  partieipants 
reported  that  partieipating  in  the  study  led  to  inereased  group  awareness  of  safety. 
While  partieipants  identified  the  need  for  major  system  ehanges — typieally 
eleetronie  health  reeords,  more  staff,  and  staff  training — they  also  felt  that 
reporting  led  to  more  immediate  benefieial  ehanges  than  paper-based  traeking 
systems.  The  latter  ineluded  inereased  awareness  of  errors  and  inereased 
diligenee.  One  partieipant  eommented,  “The  study  has  made  me  painfully  aware 
of  the  faet  that  we  don’t  have  a  system  and  seeondly,  how  diffieult  it  is  to  set  up  a 
system.”  Many  of  the  praetiees  deseribed  praetieewide  ehanges  they  made,  both 
small  and  large,  as  a  result  of  partieipating  in  the  study.  One  praetiee  deseribed 
how  it  ehanged  from  “writing  everything  down  on  little  pieees  of  paper  and 
stieking  them  on  eharts,  to  e-mailing  baek  and  forth  on  lab  results.” 

Refining  the  Web-based  reporting  system 

These  studies  have  produeed  a  Web-based  error-reporting  tool  that  eontributes 
to  our  understanding  of  threats  to  patient  safety  in  primary  eare.  It  is  not  the  only 
method  we  have  employed  for  this  purpose,  but  it  remains  an  important  and 
evolving  part  of  the  AAFP  patient  safety  researeh  program.  Future  versions  of  the 
reporting  system  will  emphasize  data  eolleeted  by  pulldown  menus  and  eheeked 
boxes,  but  some  eontinued  free-text  eomponent  is  essential. 

The  latter  two  DCERPS  studies  have  developed  the  error-eoding  eapaeity  of 
the  researeh  team  and  made  it  elear  that  timely  analysis  will  require  some  level  of 
automation,  whieh  is  another  essential  foeus  of  researeh.  Our  eoding  of  report 
data  has  most  reeently  turned  from  granularity — that  is,  using  reports  to  finitely 
deseribe  the  taxonomy  of  errors — to  funetional  aggregations  of  error  types  and 
eauses.  The  goal  is  a  taxonomy  that  ean  guide  interventions  and  make  sense  of  the 
easeades  of  errors  that  produee  adverse  outeomes  and  near  misses.  We  are  also 
leading  an  international  eonferenee  on  primary  eare  patient  safety  taxonomies  to 
better  understand  their  funetional  differenees,  the  knowledge  they  produee,  and 
how  they  may  be  eross-walked  to  eaeh  other  and  to  higher-level  elassifieations. 
Revision  of  our  primary  eare  error  taxonomy  will  likely  lead  to  reporting-tool 
modifieations.  For  a  variety  of  reasons  explained  in  the  next  seetion,  the  Web- 
based  reporting  tool  will  remain  a  researeh  tool.  Our  next  studies  will  foeus  on 
expanding  the  Web-based  proeess  to  add  a  feedbaek  funetion  for  reporters  and 
embedding  these  in  a  Web-based  quality-improvement  eollaborative. 


Legal  and  practical  barriers  to  voluntary  reporting 

Legal  barriers 

In  Table  2,  we  have  summarized  some  of  the  legal  barriers  and  eoneerns  noted 
during  our  studies,  as  well  as  the  eharaeteristies  that  safety  experts  believe  to  be 
neeessary  for  a  sueeessful  voluntary  reporting  system.^  A  major  impediment  to 
aehieving  a  health  system  version  of  the  Aviation  Safety  Reporting  System 
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Table  2.  Reporting  system  characteristics,  barriers,  results,  and  potential  future 


Reporting  system 
characteristics 

Barriers 

Results 

Future 

Nonpunitive  and 
undiscoverable 

•  Lack  of  legal 
protections  from 
discoverability 

•  Reports  are 
anonymous 

•  Not  able  to  share 
reports  with  other 
systems 

•  Lack  of  trust  by 
reporters 

•  Limited  reporting 

•  No/limited 
discoverability 

Confidential 

•  Reporter  and  data 
security  limitations 

•  Complicated  measures 
to  protect  data  and 
reporters 

•  Ongoing  concern  for 
data 

•  Tested  methods  that 
protect  reporters  from 
identification 

•  Ability  to  contact 
reporters  to  clarify 
reports 

Independent  report 
collection  and 
analysis  by  peer 
experts 

•  No  governmental 
option  obvious 

•  Lack  of  legislated 
designations  for 
specialty 
organizations 

•  Focus  difficult  to  break 
out  of  hospitals/health 
systems 

•  Lack  of  trust  by 
reporters 

•  Gathering  report  data 
without  analyses  or 
safety/quality  learning 

•  T rusted  experts  receive 
and  analyze  reports 
from  all  settings 

•  Report  analysis 
process  embedded  or 
closely  linked  to 
intervention,  education, 
and  communication 
mechanisms 

Timely  and 
responsive 

•  Lack  of  legal 
protections  from 
discoverability 

•  AAFP  team  currently 
small  and  analytic 
process  tedious 

•  Lack  of  national  or 
international 
consensus  on  analytic 
process  or 
classifications 

•  Research  juvenile 
regarding  how  to 
respond 

•  Reporters  receive  no 
direct  or  specific 
feedback 

•  No  way  to  contact 
reporters  for  more 
information  on  events 

•  No  ability  to  share 
relevant  reports  with 
other  systems 

•  Limited  learning  from 
report  data 

•  Limited  opportunities  to 
test  interventions 

•  Reporters  receive 
timely  feedback  about 
specific  events, 
targeted  tools  for 
improvement,  expert 
consultation,  peer 
comparison  reports, 
certification  or 
licensure  credits 

•  Analytic  processes  are 
fairly  automated 

•  Research  teams 
focused  on  learning 
from  reporting  system, 
doing  intervention 
studies 

•  Learning  collaboratives 
common 

Ease  of  reporting 

•  Independent, 
redundant  reporting 
systems  developing 

•  Not  embedded  in  care 
processes 

•  Burden  placed  on 
reporters  to  report 
multiple  places, 
different  formats 

•  Reporter  fatigue 

•  Reports  are 
automatically  shared 
between  systems, 
across  borders 

•  Reports  automatically 
generated  and 
embedded  in  care 
process 
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(ASRS)  is  the  lack  of  legislation  making  such  a  system  confidential  and 
nonpunitive.  Such  legislation  need  not  preempt  State  or  Federal  laws  that  require 
reporting  of  adverse  or  sentinel  health  care  events  that  result  in  serious  harm  or 
death.  These  protections  need  not  limit  or  affect  the  availability  of  error-related 
information  or  evidence  currently  available  from  other  sources,  such  as  medical 
records.  Legal  protections  for  voluntary  reporting  systems  could  complement  the 
Health  Insurance  Portability  and  Accountability  Act  of  1996  and  State  and 
Federal  peer  review  laws,  striking  a  balance  between  improving  patient  safety  and 
ensuring  accountability.  This  balance  would  achieve  the  lOM’s  aims. 

The  AAFP  has  joined  more  than  94  other  medical  organizations  in  supporting 
patient  safety  legislation  that  would  establish  federally  certified  patient  safety 
organizations  (PSOs).  PSOs  would  be  able  to  receive  error  reports  confidentially, 
protected  against  discovery  and  subpoena,  and  share  information  with  other 
certified  entities,'^’  as  described  in  Table  3.  Disappointingly,  in  each  of  the  past 
3  years.  Federal  patient  safety  legislation  offering  legal  protections  has  failed  to 
become  law.  Operating  under  AHRQ-funded  research  protocols  offers  limited 
protections  from  discovery  (so  far,  untested  in  court)  and  will  potentially  permit 
the  AAFP  to  move  beyond  anonymous  reports.  However,  these  protections  do  not 
permit  the  error-reporting  tool  to  be  used  outside  of  research  protocols  to  improve 
patient  safety.  Lack  of  protective  legislation  stymies  efforts  to  translate  our 
research  into  practice. 

The  AAFP  has  invested  significant  time  and  resources  in  obtaining  expert 
advice  about  the  limitations  of  Federal  protections,  about  other  discoverability 
risks,  and  in  maintaining  state-of-the-art  data  protection.  However,  the  AAFP 
cannot  expose  family  physicians  and  their  offices  to  the  risks  of  nonanonymous 
error  reporting  until  Federal  protections  are  in  place.  The  lOM  recognized  the 
tension  between  the  public’s  demand  for  transparency  and  the  health  system’s 
insistence  on  confidentiality  and  legal  protection  for  reported  errors.  The  lOM 
came  down  on  the  side  of  protecting  reports  out  of  concern  that  discoverability 
and  fear  of  retaliation  would  be  major  impediments  to  effective  reporting.  It 
called  for  enhanced  protection  in  order  to  establish  voluntary  reporting  systems 
and  the  development  of  a  national  patient  safety  database  of  de-identified  data.  It 
considered  this  protection  essential  for  the  integrity  and  effectiveness  of  patient 
safety  learning  systems. 

Practical  barriers 

Even  if  Federal  legislation  clears  these  hurdles,  the  practical  barriers  to  a 
national  reporting  system  could  be  enormous.  Leape  estimates  that  such  a  system 
could  attract  reports  of  0.5  to  5  million  near  misses  annually.^  This  would  be  15  to 
50  times  the  number  of  reports  to  the  ASRS;  and  if  per-case  report  review  costs 
approach  that  of  the  ASRS  (about  $70  per  case  in  2002),  the  total  cost  would  be 
staggering.  The  United  States  has  a  few  national  reporting  systems  that  are  limited 
in  their  scope,  and  Australia,  England,  and  the  Canadian  province  of 
Saskatchewan  have  or  have  announced  plans  to  introduce  national  or  provincial 
error  reporting  systems.'^’  If  the  legal  and  practical  barriers  to  national 


130 


Development  and  Legal  Issues  for  Error  Tracking 


Table  3.  General  principles  for  patient  safety  reporting  systems 


1. 

Creating  an  environment  for  safety.  There  should  be  a  nonpunitive  culture  for  reporting 
health  care  errors  that  focuses  on  preventing  and  correcting  systems  failures  and  not  on 
individual  or  organization  culpability. 

2. 

Data  analysis.  Information  submitted  to  reporting  systems  must  be  comprehensively 
analyzed  to  identify  actions  that  would  minimize  the  risk  that  reported  events  recur. 

3. 

Confidentiality.  Confidentiality  protections  for  patients,  health  care  professionals,  and 
health  care  organizations  are  essential  to  the  ability  of  any  reporting  system  to  learn  about 
errors  and  effect  their  reduction. 

4. 

Information  sharing.  Reporting  systems  should  facilitate  the  sharing  of  patient  safety 
information  among  health  care  organizations  and  foster  confidential  collaboration  with  other 
health  care  reporting  systems. 

5.  Legal  status  of  reporting  system  information.  The  absence  of  Federal  protection  for 
information  submitted  to  patient  safety  reporting  systems  discourages  the  use  of  such 
systems,  which  reduces  the  opportunity  to  identify  trends  and  implement  corrective 
measures.  Information  developed  in  connection  with  reporting  systems  should  be  privileged 
for  purposes  of  Federal  and  State  judicial  proceedings  in  civil  matters,  and  for  purposes  of 
Federal  and  State  administrative  proceedings,  including  discovery,  subpoenas,  testimony, 
or  any  other  form  of  disclosure. 

(a)  Scope.  The  privilege  for  the  information  prepared  for  a  reporting  system  should  extend 
to  any  data,  report,  memorandum,  analysis,  statement,  or  other  communication 
developed  for  the  purposes  of  the  system.  This  privilege  should  not  interfere  with  the 
disclosure  of  information  that  is  otherwise  available,  including  the  right  of  individuals  to 
access  their  own  medical  records. 

(b)  No  waiver.  The  submission  of  health  care  error  information  to  a  reporting  system,  or 
the  sharing  of  information  by  health  care  organizations  or  reporting  systems  with  third 
parties  in  accordance  with  these  principles,  should  not  be  construed  as  waiving  this 
privilege  or  any  other  privilege  under  Federal  or  State  law  that  exists  with  respect  to  the 
information. 

(c)  Freedom  of  Information  Act.  Health  care  error  information  received  by  and  from 
reporting  systems  should  be  exempt  from  the  Freedom  of  Information  Act  and  similar 
State  laws.  Such  an  exemption  is  necessary  to  preserve  the  privilege  discussed  in  this 
principle. 

(d)  Impact  on  State  law.  A  Federal  law  is  necessary  to  assure  protection  of  information 
submitted  to  national  reporting  systems,  but  the  Federal  protection  should  not  preempt 
State  evidentiary  laws  that  provide  greater  protection  than  Federal  law.  Providing  such 
information  to  reporting  systems  should  not  constitute  a  waiver  of  any  State  law 
privilege. 


voluntary  error-reporting  systems  in  the  United  States  prove  insurmountable, 
system-speeifie  or  specialty-based  reporting  systems  are  potential  solutions.^ 
These  could  feed  information  into  a  national  database. 

The  AAFP  has  identified  some  other  practical  barriers  to  sponsoring  a 
national  patient  safety  reporting  system.  Our  team  has  become  adept  and 
increasingly  reliable  in  its  coding  of  reports,  but  coding  remains  sufficiently 
tedious  that  opening  the  Web-based  reporting  system  to  the  general  membership 
of  the  AAFP  would  be  impossible  while  using  the  current  manual  coding 
mechanisms.  Most  coding  to  date  has  focused  on  the  primary  error  being  reported. 
But  some  of  the  work  done  by  us  and  others  suggests  that  more  attention  to 
cascades  of  errors  and  the  most  proximal  of  identifiable  errors  may  be  more 
important  for  identifying  potential  remedies. Cascade  and  root-cause  analyses 
will  require  considerably  more  time,  a  bigger  coding  team,  and  the  ability  to 


131 


Advances  in  Patient  Safety:  Vol.  3 


query  reporters.  Our  desire,  shared  by  many  of  our  study  partieipants,  is  to  have  a 
feedback  mechanism  that  offers  evidence-based  solutions  to  commonly  reported 
errors. 

A  final  practical  issue  for  the  near  future  is  the  need  for  specialized  expertise 
in  the  storage  and  retrieval  of  large  datasets. Medicine  has  appropriately  reached 
outside  of  itself  to  draw  on  the  skills  of  human  systems  engineers  and  aviation 
safety  experts  to  improve  reporting  systems  and  how  we  respond  to  the  errors 
they  identify.  As  future  users  of  primary  care  medical  error-reporting  data  seek  to 
mine  them  for  patterns,  or  to  analyze  them  with  data  from  other  settings  or 
systems,  complex  data  must  be  organized  to  be  easily  retrievable.  Expertise  in 
data  management  and  organization  may  reduce  these  hurdles  for  the  future,  and 
may  actually  ensure  that  reports  to  one  system  (or  specialty)  can  flow  to  other 
databases  appropriately.^^ 

The  future 

With  the  lifting  of  legal  hurdles  and,  perhaps,  with  the  implementation  of 
authorizing  legislation,  the  AAFP  could  be  designated  a  patient  safety 
organization — the  entity  to  which  participating  practices  routinely  submit  error 
reports.  Within  such  a  PSO,  voluntary  error  reporting  could  become  an  essential 
component  for  safety  and  quality  improvement  in  health  care  (Table  2).  The  goal 
would  be  to  build  reporting  and  feedback  systems  with  automated  coding  driven 
by  a  functional  error-classification  scheme  that  guides  an  evidence-driven 
feedback  mechanism.  This  system  would  also  give  reporters  access  to  the  past 
experiences  of  other  reporters  and  robust  evidence  about  implementing  systems 
that  improve  care. 

Nearly  75  percent  of  family  physicians  belong  to  the  AAFP  and  look  to  it  as  a 
trusted  entity  for  improving  the  quality  of  the  care  they  provide.  Our  studies 
suggest  that  members  would  make  error  reports  to  the  AAFP,  and  when  they  did, 
they  would  want  specific  feedback  to  improve  their  performance.  The  AAFP 
could  link  reports  and  their  analyses  to  a  wider  array  of  its  existing  programs  to 
improve  quality  and  safety.  As  a  formal  PSO,  the  AAFP  could  analyze  reports, 
provide  feedback  to  reporting  practices,  and  provide  tools  for  preventing  similar 
errors,  allowing  participating  practices  to  monitor  their  progress  relative  to  peer- 
practices  and  to  share  relevant  reports  with  other  safety  organizations.  The  AAFP 
could  tailor  its  existing  educational  arm  to  reflect  report  findings,  produce  quality 
improvement  publications  and  “toolboxes,”  e-mail  “patient  safety  cases  of  the 
week”  to  its  membership,  and  provide  learning  collaboratives.  Physician 
participation  would  be  linked  to  professional  development,  licensing 
requirements,  and  maintenance  of  certification.  The  AAFP’s  investments  and 
vision  expose  a  desire  to  combine  its  trusted  relationship  with  members,  its  well- 
established  educational  infrastructure,  and  its  new  capacities  to  position  itself  as  a 
catalyst  for  improving  quality  in  primary  care.  Other  medical  professional 
organizations  could  similarly  support  their  members. 

The  AAFP  error-reporting  system  would  become  a  part  of  routine  research 
and  learning.  If  the  legal  and  practical  barriers  can  be  overcome,  the  AAFP  could 
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join  Federal  and  speeialty  partners,  health  systems,  and  other  countries  in  doing 
patient  safety  research.  The  international  LINNAEUS  collaboration  and  the 
AAFP’s  leadership  in  exploring  how  its  error  taxonomy  links  to  other  patient 
safety  classifications  hint  at  this  future.  Outside  of  research,  sharing  report  data 
across  systems  and  borders  could  facilitate  more  robust  learning.  Medication  error 
reports  from  primary  care,  for  example,  could  be  repackaged  and  shared  with  the 
Food  and  Drug  Administration  and  other  medication  error-reporting  processes. 


Conclusion 

Over  the  past  5  years,  the  American  Academy  of  Family  Physicians  has 
developed  and  tested  a  Web-based,  voluntary  error-reporting  system  through  a 
series  of  domestic  and  international  studies.  This  research  effort  has  produced  an 
error-reporting  process  that  is  accepted  and  used  by  physicians  and  clinic  staff. 
The  reporting  system  feeds  into  a  robust  classification  process  that  is  revealing 
where  things  go  wrong  in  primary  care,  how  errors  differ  from  inpatient  settings, 
and  potential  ways  to  make  care  safer.  Patient  safety  data  are  a  critical  input  to  the 
efforts  of  providers  to  redesign  care  processes  in  ways  that  will  make  health  care 
safer  for  all  patients.^ 

Our  qualitative  data  reveal  that  participation  in  reporting  errors  can  induce 
physicians  and  their  staffs  to  modify  error-prone  systems  in  their  practices. 
Currently,  a  chasm  of  legal  and  practical  problems  exists  between  the  collection 
of  error  reports  and  providing  constructive  feedback  to  reporting  providers  to 
assist  them  in  improving  the  quality  and  safety  of  their  offices.  These  gaps  require 
more  study.  Our  research  will  move  from  anonymous  reports  to  confidential 
reports  under  the  legal  protections  afforded  by  Federal  grant  funding,  so  that  we 
may  begin  testing  direct  feedback  to  reporters,  doing  root-cause  analysis,  and 
engaging  the  other  education  and  dissemination  programs  of  the  AAFP.  Whether 
this  Web-based  error-reporting  system  evolves  beyond  research,  and  whether  the 
AAFP  becomes  a  full-fledged  change  agent  for  primary  care  practice,  depends  on 
bridging  the  legal  and  practical  chasms  identified  by  the  lOM  and  others. 
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